Whitelisting Onelake API & Workspace PL FQDNs #552

SmritiAgrawal04 · 2025-11-25T09:13:36Z

Which issue does this PR close?

Closes #.

Rationale for this change

With the release of the WS PL FQDNs with workspace private links, we need to add support for the new workspace FQDNs (“{wsid}.z{xy}.dfs.fabric.microsoft.com), so users can continue to use Delta-RS with OneLake and workspace private links.
We have also added support for "-api.onelake" FQDNs. as well Ref.

What changes are included in this PR?

Valid WS-PL FQDN egs:

Invalid WS-PL FQDN egs:

https://c047b3e34e89407a98d7cf9949ae92a3.z12.blob.fabric.microsoft.com // xy mismatch
https://c047b3e34e89407a98d7cf9949ae923.z12.blob.fabric.microsoft.com // workspaceid 31 chars

New API (regional & non-regional) endpoint support

Are there any user-facing changes?

Yes. The documentaion is as below:

alamb

Thank you @SmritiAgrawal04

This PR needs some tests to show it working I think

crepererum

I do somewhat agree with @alamb. I know it's close to impossible to write an integration test for this, but maybe we can at least have a unit test for parse_url?

kashyap-kunal · 2025-12-22T05:20:05Z

src/azure/builder.rs

+                    let first_label = host.split('.').next().unwrap_or_default();
+                    self.account_name = Some(validate(first_label)?);
+
+                    let container = parsed.path_segments().unwrap().next().expect(


nit: onelake uses workspace terminology

kashyap-kunal · 2025-12-22T05:24:13Z

src/azure/builder.rs

+                // Regex to match WS-PL FQDN: "{workspaceid}.z??.dfs.fabric.microsoft.com"
+                // workspaceid = 32 hex chars, z?? = z + first two chars of workspaceid
+                lazy_static::lazy_static! {
+                    static ref WS_PL_REGEX: Regex = Regex::new(r"^(?P<workspaceid>[0-9a-f]{32})\.z(?P<xy>[0-9a-f]{2})\.(dfs|blob)\.fabric\.microsoft\.com$").unwrap();


Let's add support for .onelake.fabric.microsoft.com also

kashyap-kunal · 2025-12-22T06:02:14Z

src/azure/builder.rs

+                    let xy = captures.name("xy").unwrap().as_str();
+
+                    // Validate z?? matches first 2 chars of workspaceid
+                    if &workspaceid[0..2] != xy {


remove this validation

kashyap-kunal · 2025-12-22T06:07:51Z

src/azure/builder.rs

+                }
+
+                // Otherwise, check Fabric global / Onelake API FQDN 
+                if host.ends_with(DFS_FABRIC_SUFFIX) || host.ends_with(BLOB_FABRIC_SUFFIX) {


if we are checking for global endpoint, should these not be with not like (!host.ends_with(DFS_FABRIC_SUFFIX) && !host.ends_with(BLOB_FABRIC_SUFFIX))

kashyap-kunal · 2025-12-22T06:18:50Z

src/azure/builder.rs

+                if host.ends_with(DFS_FABRIC_SUFFIX) || host.ends_with(BLOB_FABRIC_SUFFIX) {
+                let labels: Vec<&str> = host.split('.').collect();  
+                let account_name = if labels.len() >= 2 && labels[0].contains("api") && labels[1] == "onelake" {
+                                        format!("{}-{}", labels[0], labels[1])


If we are referring workspace id as account_name, then this will not work for non pl scenario. In those case it will give account_name as "westus-api-onelake"

SmritiAgrawal04 · 2026-01-09T05:04:33Z

Hi @alamb & @crepererum,

I added the unit tests as suggested. I request to review the PR please.

src/azure/builder.rs

alamb · 2026-01-12T16:39:25Z

src/azure/builder.rs

+                    self.container_name = Some(validate(parsed.username())?);
+                    self.account_name = Some(validate(a)?);
+                    self.use_fabric_endpoint = true.into();
+                } else if let Some(a) = host.strip_suffix("-api.onelake.fabric.microsoft.com") {


Is *-api.onelake.fabric.microsoft.com a publicly documented endpoint? If yes, can you point to the Microsoft doc so we can cite it in code/tests?

I don't see it in https://learn.microsoft.com/en-us/fabric/onelake/onelake-access-api

This comment is still outstanding I think. Is this a vaild url?

Yeah it is a valid URL- added in the documentation.

alamb

Thanks @SmritiAgrawal04 -- i left some more questions

BTW I asked codex for a review of this PR like this

please review this PR (checked out in this directory): https://github.com/apache/arrow-rs-object-store/pull/552

  Specifically ensure that the format of the URLs supported matches publically published microsoft urls and provide documentation references

And this was the output which I found helpful

───────────────────────────────────────────────────────────────────────────────────────────────

• Findings

High: -api.onelake.fabric.microsoft.com is accepted for both abfs(s) and https, but I can’t find this FQDN in
the public OneLake URL docs. The published endpoints are onelake.dfs.fabric.microsoft.com and workspace-scoped
https://{workspaceId}.z{xy}.dfs.fabric.microsoft.com/... (see https://learn.microsoft.com/en-us/fabric/onelake/
onelake-access-api). If -api.onelake is internal or not publicly documented, this should be gated or removed.
Medium: The WS‑PL regex no longer verifies that z{xy} matches the first two hex chars of the workspace id (src/
azure/builder.rs:692-708). The published WS‑PL format is https://
{workspaceId}.z{first2}.dfs.fabric.microsoft.com (same doc). Without the check, non‑published FQDNs are
accepted.
Medium: WS‑PL regex allows .onelake.fabric.microsoft.com (src/azure/builder.rs:692-699). The public docs list
dfs (and blob endpoints for blob access), but I don’t see onelake as a WS‑PL subdomain. Please confirm with a
Microsoft reference or remove to match published formats.
Low: WS‑PL regex only allows lowercase hex ([0-9a-f]{32}) (src/azure/builder.rs:697). If Microsoft treats
workspace IDs as case‑insensitive GUIDs in URLs, this will reject valid uppercase forms; consider documenting
that only lowercase is supported or make the regex case‑insensitive.

Doc references

OneLake access API URL formats: https://learn.microsoft.com/en-us/fabric/onelake/onelake-access-api
Private Link / workspace-specific OneLake endpoints (if this PR targets WS‑PL): please confirm the exact doc
section that defines the workspaceId.z{xy}.dfs.fabric.microsoft.com pattern and whether any onelake WS‑PL host
is documented (I could not find it in public docs).

Questions / assumptions

Is *-api.onelake.fabric.microsoft.com a publicly documented endpoint? If yes, can you point to the Microsoft
doc so we can cite it in code/tests?
Should WS‑PL accept only dfs/blob subdomains, or is .onelake.fabric.microsoft.com explicitly published?

tustvold · 2026-01-14T20:40:19Z

FYI #604 may be related.

SmritiAgrawal04 · 2026-01-16T05:51:37Z

Thanks @SmritiAgrawal04 -- i left some more questions

BTW I asked codex for a review of this PR like this
please review this PR (checked out in this directory): https://github.com/apache/arrow-rs-object-store/pull/552

  Specifically ensure that the format of the URLs supported matches publically published microsoft urls and provide documentation references
And this was the output which I found helpful

───────────────────────────────────────────────────────────────────────────────────────────────

• Findings

High: -api.onelake.fabric.microsoft.com is accepted for both abfs(s) and https, but I can’t find this FQDN in
the public OneLake URL docs. The published endpoints are onelake.dfs.fabric.microsoft.com and workspace-scoped
https://{workspaceId}.z{xy}.dfs.fabric.microsoft.com/... (see https://learn.microsoft.com/en-us/fabric/onelake/
onelake-access-api). If -api.onelake is internal or not publicly documented, this should be gated or removed.

Medium: The WS‑PL regex no longer verifies that z{xy} matches the first two hex chars of the workspace id (src/
azure/builder.rs:692-708). The published WS‑PL format is https://
{workspaceId}.z{first2}.dfs.fabric.microsoft.com (same doc). Without the check, non‑published FQDNs are
accepted.

Medium: WS‑PL regex allows .onelake.fabric.microsoft.com (src/azure/builder.rs:692-699). The public docs list
dfs (and blob endpoints for blob access), but I don’t see onelake as a WS‑PL subdomain. Please confirm with a
Microsoft reference or remove to match published formats.

Low: WS‑PL regex only allows lowercase hex ([0-9a-f]{32}) (src/azure/builder.rs:697). If Microsoft treats
workspace IDs as case‑insensitive GUIDs in URLs, this will reject valid uppercase forms; consider documenting
that only lowercase is supported or make the regex case‑insensitive.

Doc references

OneLake access API URL formats: https://learn.microsoft.com/en-us/fabric/onelake/onelake-access-api

Private Link / workspace-specific OneLake endpoints (if this PR targets WS‑PL): please confirm the exact doc
section that defines the workspaceId.z{xy}.dfs.fabric.microsoft.com pattern and whether any onelake WS‑PL host
is documented (I could not find it in public docs).

Questions / assumptions

Is *-api.onelake.fabric.microsoft.com a publicly documented endpoint? If yes, can you point to the Microsoft
doc so we can cite it in code/tests?

Should WS‑PL accept only dfs/blob subdomains, or is .onelake.fabric.microsoft.com explicitly published?

Hi @alamb,

I have addressed all comments. About, finding #1 & #3, we plan to add it to the public documentation, the PR for which is already out. I request to approve these changes meanwhile. Thanks

alamb

Thank you for your work on this @SmritiAgrawal04

I am sorry for the length of time but our review bandwdith is very limited here

Part of the reason it has taken so long to review is there is no ticket or description of what issue this is fixing, nor good examples of what new features you have added

In order to proceed, please:

Fill out the PR description with a clear description of what problem you are solving. This shoud, at a minimum,, provide URLs that can't be parsed with the current crate, and examples of the expected output
Provide a link to the documentation of the format of onelake URLs so that we can verify the URLs should be parsed
Add a link to the docs / references in the code so that future readers can refer back to them

alamb · 2026-01-17T15:51:10Z

src/azure/builder.rs

+                    self.container_name = Some(validate(parsed.username())?);
+                    self.account_name = Some(validate(a)?);
+                    self.use_fabric_endpoint = true.into();
+                } else if let Some(a) = host.strip_suffix("-api.onelake.fabric.microsoft.com") {


This comment is still outstanding I think. Is this a vaild url?

Cargo.toml

alamb · 2026-01-17T15:56:19Z

src/azure/builder.rs

+        builder
+            .parse_url("https://Ab000000000000000000000000000000.zAb.dfs.fabric.microsoft.com/")
+            .unwrap();
+        assert_eq!(builder.account_name, Some("ab000000000000000000000000000000.zab".to_string()));


is the account name really supposed to have the ab000000000000000000000000000000 in it? It seems confusing that the container name is also ab000000000000000000000000000000

alamb · 2026-01-17T15:57:32Z

src/azure/builder.rs

+
+                let ws_pl_regex = WS_PL_REGEX.get_or_init(|| {
+                    Regex::new(
+                        r"(?i)^(?P<workspaceid>[0-9a-f]{32})\.z(?P<xy>[0-9a-f]{2})\.(onelake|dfs|blob)\.fabric\.microsoft\.com$"


What is the rationale for using regexp math here rather than just hostname splitting? The examples / tests you have added don't seem to test for invalid URLs / that don't follow a simple .... format

Workspace private link FQDNs follow a pattern unlike other FQDNs that had been whitelisted earlier. WS‑PL hostnames have a very specific, structured format with invariants that we want to validate, not merely parse. If you verify the FQDN syntax here , you'd note that a simple split('.') can tell “there are N labels” and extract strings, but it doesn’t naturally enforce:

wsid is exactly 32 hex chars,

xy is exactly 2 hex chars,

and xy == wsid[0..2].

Added the test cases for invalid ws-pl fqdn.

alamb · 2026-01-17T16:01:24Z

I have addressed all comments. About, finding #1 & #3, we plan to add it to the public documentation, the PR for which is already out. I request to approve these changes meanwhile. Thanks

Some comments remain unaddressed. In terms of documentation, please include it in this PR

kevinjqliu · 2026-01-19T17:58:12Z

Hey folks, thanks for taking the time to review this PR.
I work on OneLake. @SmritiAgrawal04 and I will sync offline to clean up a PR before requesting another round of review

Whitelisting Onelake API & Workspace PL FQDNs

3960a7a

alamb reviewed Nov 25, 2025

View reviewed changes

crepererum reviewed Dec 12, 2025

View reviewed changes

tustvold marked this pull request as draft December 13, 2025 13:22

kashyap-kunal suggested changes Dec 22, 2025

View reviewed changes

Addressing comments to whitelist api-onelake fqdns and add UTs

5b9dd47

kashyap-kunal approved these changes Jan 8, 2026

View reviewed changes

SmritiAgrawal04 marked this pull request as ready for review January 9, 2026 05:03

crepererum reviewed Jan 9, 2026

View reviewed changes

src/azure/builder.rs Outdated Show resolved Hide resolved

Reverting the mistakenly modified UT

4ba4321

SmritiAgrawal04 closed this Jan 12, 2026

SmritiAgrawal04 reopened this Jan 12, 2026

alamb reviewed Jan 12, 2026

View reviewed changes

src/azure/builder.rs Show resolved Hide resolved

alamb reviewed Jan 12, 2026

View reviewed changes

src/azure/builder.rs Show resolved Hide resolved

alamb reviewed Jan 12, 2026

View reviewed changes

SmritiAgrawal04 closed this Jan 13, 2026

Adding validation for xy in WS-PL URL & case insensitive regex

3826e5b

SmritiAgrawal04 reopened this Jan 13, 2026

alamb reviewed Jan 17, 2026

View reviewed changes

Making regex an optional dependency

897b528

Whitelisting Onelake API & Workspace PL FQDNs #552

Are you sure you want to change the base?

Whitelisting Onelake API & Workspace PL FQDNs #552

Uh oh!

Conversation

SmritiAgrawal04 commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

crepererum left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SmritiAgrawal04 commented Jan 9, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tustvold commented Jan 14, 2026

Uh oh!

SmritiAgrawal04 commented Jan 16, 2026

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevinjqliu commented Jan 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

SmritiAgrawal04 commented Nov 25, 2025 •

edited

Loading

alamb left a comment •

edited

Loading

alamb commented Jan 17, 2026 •

edited

Loading